GPU Enabling and Usage

Table of Contents [Sticky]

Use the IQ-9075 GPU for supported graphics, composition, and compute workloads through the APIs exposed by the installed Adreno driver, such as OpenGL ES, Vulkan, and OpenCL. This guide provides verified example pipelines to leverage the GPU for image processing through the OpenGL GStreamer elements, by applying effects and filters like the Sobel filter. The performance results show that the GPU utilization scales with image resolution, for example, a pipeline that captures video from camera, uploads the video to GPU and applies the Sobel filter shows a 3% GPU load for 1280x720@30fps video, 6% for 1920x1080@30fps and 25% for 4K@30fps video. Use this wiki as starting point to build GPU accelerated video pipelines.

Problems running the pipelines shown on this page? Please see our GStreamer Debugging guide for help .

Introduction

These elements were tested both in the Ubuntu downloaded sources (Ubuntu 24.04.4) and the Yocto built image. GPU use was verified within the debugging logs generated from GStreamer for multiple pipelines with OpenGL elements and the following output confirms it:

GL_VENDOR: Qualcomm 
GL_RENDERER: Adreno (TM) 663

To verify the GPU usage, run the following test pipeline:

GST_DEBUG=glcontext:INFO GST_DEBUG_FILE=/tmp/gst.log gst-launch-1.0 videotestsrc is-live=1 num-buffers=30 ! glupload ! gl-test-element ! gldownload ! fakesink && cat /tmp/gst.log | grep GL_

Change the gl-test-element accordingly and verify both the GL_VENDOR and GL_RENDERER properties. There should be a similar output to the previous output mentioned.

Testing OpenGL elements

Relevant GPU-related elements inclued: glupload, gleffects, gltestsrc, gloverlay, glcolorconvert.

Simple test source and sink

 gst-launch-1.0 -v gltestsrc ! glimagesink

Expected result

GPU-generated pattern

Basic GPU upload and display

gst-launch-1.0 videotestsrc ! glupload ! glimagesink

Expected result

Test pattern displayed, stable playback, no crash

GPU color conversion

gst-launch-1.0 videotestsrc ! glupload ! glcolorconvert ! glimagesink

Expected result

Correct colors, stable playback, no visible corruption

GPU shader effect: sepia

gst-launch-1.0 videotestsrc ! glupload ! gleffects effect=sepia ! glimagesink

Expected result

Sepia effect visible, stable playback, no corruption

GPU shader effect: sobel

gst-launch-1.0 videotestsrc ! glupload ! gleffects effect=sobel ! glimagesink

Expected result

Edge detection visible, stable playback, no corruption

Chained GPU effects

gst-launch-1.0 videotestsrc ! glupload ! gleffects effect=blur ! gleffects effect=mirror ! glimagesink

Expected result

Blur and mirror effects visible, stable playback, no corruption

GPU overlay path

gst-launch-1.0 videotestsrc pattern=smpte ! glupload ! gloverlay location=logorr.png offset-x=40 offset-y=40 ! glimagesink

Expected result

Video displayed with overlay path active, stable playback, no corruption, logo on top

Upload to download path

gst-launch-1.0 videotestsrc ! glupload ! gldownload ! fakesink

Expected result

Pipeline runs without errors, stable buffer flow, no crash

High-load OpenGL effects stress

gst-launch-1.0 videotestsrc ! glupload ! gleffects effect=blur ! gleffects effect=sobel ! gleffects effect=mirror ! glimagesink

Expected result

Multiple effects visible, stable playback, no freeze or corruption

Max-rate rendering check

gst-launch-1.0 videotestsrc ! glupload ! glimagesink sync=false

Expected result

Fast rendering, smooth playback, no frame stalls

Performance measurements

GL Test Source performance

The following table summarizes performance results for gltestsrc at different output resolutions. Memory usage was collected from ps (RSS), GPU utilization was derived from /sys/class/kgsl/kgsl-3d0/gpubusy as busy / total * 100, and throughput metrics were collected with the perf element using the following pipeline:

gst-launch-1.0 -v gltestsrc ! "video/x-raw(memory:GLMemory),format=RGBA,width=$WIDTH,height=$HEIGHT" ! perf print-cpu-load=true ! fakesink

Resolution	%MEM	RSS (KB)	GPU Utilization (%)	Mean FPS	CPU (%)
1280x720	0.2	97620	12.38	1728.592	18
1920x1080	0.2	97768	30.32	1568.784	13
3840x2160	0.2	98008	99.95	1508.799	13

Gl Effects performance

The following table summarizes performance results for gleffects at different resolutions and input formats. For formats I420 and NV12, a videoconvert was needed for cap negotiation

effect=sobel

With videotestsrc

gst-launch-1.0 videotestsrc ! video/x-raw,width=$WIDTH,height=HEIGHT,format=RGBA ! glupload ! gleffects effect=sobel ! perf print-cpu-load=true ! fakesink

Resolution	%MEM	RSS (KB)	GPU Utilization (%)	Mean FPS	CPU (%)
1280x720	0.3	112648	12.39	163.991	15
1920x1080	0.3	130632	17.97	114.968	15
3840x2160	0.6	228156	33.75	49.418	15

With camera

gst-launch-1.0 -v -e qtiqmmfsrc camera=0 ! 'video/x-raw,format=NV12,width=1280,height=720,framerate=30/1' ! queue max-size-buffers=4 leaky=downstream ! glupload ! glcolorconvert ! gleffects effect=sobel ! perf print-cpu-load=true ! fakesink

Resolution	%MEM	RSS (KB)	GPU Utilization (%)	Mean FPS	CPU (%)
1280x720	0.2	106928	3.03	29.812	14
1920x1080	0.3	108944	6.45	30.098	14
3840x2160	0.3	117744	25.28	30.043	14

effect=blur

With videotestsrc

gst-launch-1.0 videotestsrc ! video/x-raw,width=$WIDTH,height=HEIGHT,format=RGBA ! glupload ! gleffects effect=blur ! perf print-cpu-load=true ! fakesink

Resolution	%MEM	RSS (KB)	GPU Utilization (%)	Mean FPS	CPU (%)
1280x720	0.3	112524	13.27	145.456	14
1920x1080	0.3	130068	25.54	118.819	15
3840x2160	0.6	226716	40.88	58.632	14

With camera

gst-launch-1.0 -v -e qtiqmmfsrc camera=0 ! 'video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1' ! queue max-size-buffers=4 leaky=downstream ! glupload ! glcolorconvert ! gleffects effect=sobel ! perf print-cpu-load=true ! fakesink

Resolution	%MEM	RSS (KB)	GPU Utilization (%)	Mean FPS	CPU (%)
1280x720	0.2	105920	3.48	30.080	14
1920x1080	0.2	107524	7.54	30.079	13
3840x2160	0.3	118344	28.59	30.002	15

effect=blur + effect=sobel

With videotestsrc

gst-launch-1.0 videotestsrc ! video/x-raw,width=$WIDTH,height=HEIGHT,format=RGBA ! glupload ! gleffects effect=blur ! gleffects effect=sobel ! perf print-cpu-load=true ! fakesink

Resolution	%MEM	RSS (KB)	GPU Utilization (%)	Mean FPS	CPU (%)
1280x720	0.3	113040	16.63	101.118	15
1920x1080	0.3	131016	35.66	98.014	15
3840x2160	0.6	228592	60.45	48.459	15

With camera

gst-launch-1.0 -v -e qtiqmmfsrc camera=0 ! 'video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1' ! queue max-size-buffers=4 leaky=downstream ! glupload ! glcolorconvert ! gleffects effect=blur ! gleffects effect=sobel ! perf print-cpu-load=true ! fakesink

Resolution	%MEM	RSS (KB)	GPU Utilization (%)	Mean FPS	CPU (%)
1280x720	0.2	107496	5.37	30.197	15
1920x1080	0.3	108476	11.65	30.200	14
3840x2160	0.3	117652	45.38	30.021	15

Element latency

The following table presents the average latency for the elements in the pipeline from the subsection above.

Element Latency for sobel effect in 720p
element	count	avg_ms	min_ms	max_ms	p50_ms	p90_ms	p95_ms	p99_ms
glcolorconvertelement0	84	2.362	0.671	39.794	1.895	2.534	2.631	3.086
queue0	84	1.905	0.122	71.153	0.597	0.722	0.821	43.013
gleffectsgeneric0	84	1.655	0.425	48.884	0.825	1.713	1.748	2.297
gluploadelement0	84	1.538	0.430	5.672	1.572	1.782	2.095	3.194
capsfilter0	84	0.080	0.032	0.129	0.082	0.089	0.092	0.122
perf0	84	0.075	0.023	0.424	0.060	0.085	0.098	0.389

The following table presents the buffer total average latency of the pipeline subsection from the moment its delivered by the camera source pad until it leaves the perf source pad. Note that this is not glass to glass latency.

Total Average Summary
metric	value
TOTAL_AVG_SUM_MS	7.615
TOTAL_AVG_SUM_NS	7615400

FAQ

How can I verify if the GPU is being used?: Check CPU and GPU usage. Also set the following variables before the pipeline GST_DEBUG=glcontext:INFO GST_DEBUG_FILE=/tmp/gst.log and check for messages like "GL_VENDOR: Qualcomm " and "GL_RENDERER: Adreno (TM) 663"
Why can a GStreamer GL pipeline use high CPU?: CPU conversion, GL upload/download, synchronization, or a software renderer can dominate. Inspect negotiated memory, and transfer cost.

Related pages

❯

Share This Page

Send Email Using